Unary TEI Elements and the Token Based Corpus

The establishment of TEI as a standard for textual data generated outside of the narrow domain of corpus linguistics in history, literature, philosophy and more, has led to a fruitful integration of encoding vocabulary from different fields of interest, but at a necessary cost of a large stock of elements, heterogeneous interpretations of those elements, and limitations on the kinds of annotation combinations that a schema allows. Meanwhile in corpus and computational linguistics circles, advances in the direction of generic, vocabulary agnostic graph based models of corpus representation have gained prominence (notable examples are PAULA, Dipper 2005 and GrAF, Ide & Suderman 2007, the latter recently canonized as part of the LAF standard in ISO 24615). Graph based annotation formats lend themselves to generic, reusable query architectures, but reduce all data to having the same ontological status. Specifically, corpora in corpus linguistics center on the concept of tokens, minimal technical units of linguistic analysis, which serve as textual anchors for higher annotations (either features of the tokens, like parts of speech, or higher structures, such as syntax trees). In this paper we would like to point out a specific subset of problems caused by this dissonance between the TEI model and the token-based corpus annotation graph. We will focus on the interpretation of unary XML elements, such as line or page breaks (e.g. , ), and the representation of the underlying data structure in non-XML-based corpus query systems. Unary elements present a particular challenge for a token based corpus, since they occur within the plain text of a TEI document, yet they cover no part of the text, as shown in Figure 1.